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W e don’t always mean what we say. We 
don’t always fully cooperate when inter¬ 
acting with others. Sometimes, it’s convenient to 
lie or omit the truth. We might not be fully honest 

to a conversational partner, for example, because 
we don’t want to hurt his or her feelings. Many 
daily life interactions are negotiations. We have 
certain goals or preferences, and we want to re¬ 
alize them, often without being explicit. Written 
communication doesn’t really differ from these as¬ 
pects of face-to-face interactions. All this happens 
many times each day . 1 

Human-computer interaction examines how 
humans interact with computers and looks at new 
technology that enables such interactions to be 
more efficient, more convenient, or more enter¬ 
taining. New interaction technology uses all kinds 
of sensors; rather than being attached to PCs, such 
sensors disappear in the environment, into walls, 
furniture, clothes, toys, personal mobile devices, 
and mobile devices in the environment, such as 
social robots that help humans. Hence, human- 
computer interaction has become human-media or 
human-environment interaction, where the envi¬ 
ronment supports a human inhabitant in his or her 
activities. Providing the best possible support re¬ 
quires understanding and anticipating natural hu¬ 
man behavior and not just the explicit commands 
that are issued to get a certain task done. When 
multiple users inhabit the environment, it requires 
understanding human-human or multiparty in¬ 
teraction. Members in such a party can be social 
robots or autonomous agents that can sometimes 
take the form of virtual humans that display ver¬ 
bal and nonverbal behavior. 

In such an environment, looking at natural inter¬ 
action behavior will then include looking at non- 
cooperative and misleading behavior and deciding 
how to deal with it. Obviously, this is particu¬ 
larly true if we must understand human-human 


interaction in such environments and if we try to 
model natural interaction between a human and 
a social robot or virtual human. But, of course, 
we’re interested in understanding, modeling, de¬ 
signing, and displaying misleading and noncoop¬ 
erative behavior in many more situations and, in 
particular, applications. 

In a mixed-reality training and simulation envi¬ 
ronment equipped with cameras, microphones, po¬ 
sition sensors, and maybe sensors that gather physi¬ 
ological information from a trainee, we might want 
to enter misleading information, enable miscommu- 
nication, and include virtual agents’ noncooperat¬ 
ing behavior to make it more realistic . 2 In a sports 
simulation environment, we might want to train a 
player on a virtual opponent’s misleading actions 
(feints) or vice versa — that is, try to mislead the 
virtual opponent . 3 In a more traditional educa¬ 
tional environment, a virtual teacher can express 
disappointment or satisfaction when a need exists 
to stimulate a student, although these might not be 
the emotions the teacher feels . 4 Therapy robots for 
treating autism have been introduced that simulate 
emotions to help develop communication skills . 5 

Such tools for developing social interaction 
skills need to know about desired behavior as well 
as undesired behavior. In multi-agent negotiation 
systems, we can send our agent away to negotiate, 
and, by definition, the agent shouldn’t start nego¬ 
tiating by revealing its strategy and what its final 
offer will be. This must be kept secret. Agents in 
simulation and negotiation situations can have se¬ 
crets. And virtual humans or socially intelligent 
robots that act in social situations must be care¬ 
ful in revealing information from their interaction 
partners to gain and maintain their trust. 

Modeling and Detecting Deception 

Many applications requiring knowledge about 
how to deceive are related to safety, security, 
and warfare. Speech and text analysis can help 
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detect deception in speech and text. 
Cameras, microphones, physiological 
sensors, and intelligent software can 
detect behavioral cues that identify 
misleading and suspicious activity 
in public spaces. 6 Models of decep¬ 
tion and noncooperation can make 
a virtual or mixed-reality training 
environment more realistic, improve 
immersion, and thus make it more 
suitable for training military or se¬ 
curity personnel. In more serious ap¬ 
plications, we can have robots that 
operate in physical and nontraining 
environments where they must per¬ 
form military activity, including mis¬ 
leading the enemy (such as enemy 
robots) or where they’re involved in 
rescue operations. In the latter situ¬ 
ation, omitting the truth could help 
prevent a panic situation that would 
lead to more victims. 7 

Deception in digital games and en¬ 
tertainment applications is natural, 
whether we’re talking about chess 
or a role-playing video game. In par¬ 
ticular, when agents in game and en¬ 
tertainment situations become more 
intelligent, more autonomous, and 
more emotional, misleading the op¬ 
ponent becomes part of the game. 
When the human player has no more 
secrets from his or her computer op¬ 
ponent, 8 the empathetic computer 
must determine how interesting or 
humiliating the game will be for its 
human opponent. 

Deception isn’t a new issue in com¬ 
puter science or in human-computer 
interaction. In fact, the Turing test is 
about deception. It works both ways: 
the computer tries to hide not be¬ 
ing human, while its human conver¬ 
sational partner attempts to detect 
and exploit weak parts of the com¬ 
puter’s intelligence by misleading it 
with trick questions; the human cer¬ 
tainly isn’t cooperative. Similarly, 
Joseph Weizenbaum’s well-known 
Eliza program is about deception. 
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Finally, I should mention the behavior 
of HAL in Stanley Kubrick’s 2001: 
A Space Odyssey (1968). In a criti¬ 
cal situation during a space mission, 
the astronauts have doubts about the 
decisions the spaceship’s computer 
(HAL) has made. They try to hide 
their intentions to overrule HAL, but 
the computer has sensors that let it 
monitor their conversations and an¬ 
ticipate their actions. However, in the 
end, the astronauts are more clever 
in misleading HAL than the other 
way around. What happens when 
the computer doesn’t obey us, or we 
don’t obey it? 9 

Here, I’ve surveyed many aspects 
of computational deception and 
noncooperative behavior. Some as¬ 
pects are still missing. For example, 
what is the relation between this de¬ 
ception and noncooperative behav¬ 
ior research and the many attempts 
to model humor in a computational 
way? What role do emotions (affect) 
play as regards the topics we’ll dis¬ 
cuss? Considerable literature is also 
available on lying and lie detection, 10 
and on noncooperative interaction 
behavior 11 and deceptive agents. 12 

In This Issue 

In the six contributions to this in¬ 
stallment of Trends & Controversies, 
we find state-of-the-art research ap¬ 
proaches to the analysis and genera¬ 
tion of noncooperative and deceptive 
behavior in virtual humans, agents, 
and robots; the analysis of multiparty 
interaction in the context of decep¬ 
tive behavior; and methods to detect 
misleading information in texts and 
computer-mediated communication. 

Ronald Arkin discusses modeling 
deception within a group of robotic 
agents. He looks at models from social 
psychology and cognition (partner 
modeling), but also introduces biolog¬ 
ically inspired models of deception, 
looking at squirrels’ food-protecting 

www.computer.org/intelligent 


strategies and bird mobbing behav¬ 
ior. The research from Sebastien 
Brault and his colleagues exam¬ 
ines sports and deception. It doesn’t 
really consider models of deception, 
but rather reports on experiments 
with novices and experts about 
presenting bodily deceptive cues 
in virtual reality sports training en¬ 
vironments. Players can be trained 
to detect and understand deceptive 
movements and to perform them. The 
third contribution, by David Traum, 
covers deceptive and noncooperative 
behavior in virtual reality environ¬ 
ments, where embodied conversa¬ 
tional agents interact and negotiate 
with human partners about certain 
tasks and interests. He discusses cre¬ 
ating such agents and application ar¬ 
eas. Hayley Hung’s contribution re¬ 
turns to the detection of deception’s 
nonverbal signals (from speech and 
movements), but in a multiparty con¬ 
text where more than one deceiver 
might be present. Eugene Santos and 
his colleagues emphasize the role 
and the modeling of deceptive intent 
when modeling deceptive behavior 
and detecting it via human reasoning. 
Finally, Lina Zhou and Dongsong 
Zhang discuss online deception—that 
is, deception in computer-mediated 
text-based communication and how 
we can identify from texts using nat¬ 
ural language processing and ma¬ 
chine learning techniques. 

This installment of Trends & Con¬ 
troversies provides different view¬ 
points on modeling deception. It 
also makes clear that deception re¬ 
searchers with different viewpoints, 
different theoretical and empirical 
approaches, and having different ap¬ 
plications in mind can learn from 
each other. 
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Robots that Need to 
Mislead: Biologically- 
Inspired Machine 
Deception 

Ronald C. Arkin, Georgia Institute 
of Technology 

The Georgia Tech Mobile Robot 
Laboratory has conducted consider¬ 
able research on deception and its ap¬ 
plication within robotic systems for 
the US Navy. Here, I review three 
areas: using psychological interde¬ 
pendence theory as the basis for pro¬ 
ducing deception in robotic systems 
to evade capture; studying deception 
in squirrel hoarding as a means for 
misleading a predator regarding hid¬ 
den cached resources; and mimicking 
bird mobbing behaviors as they ap¬ 
ply to deceptive activity to assess the 
value and risks associated in feigning 
strength when none exists. 

Deception has many definitions. 
The one I use to frame the rest of this 
discussion is “a false communication 
that tends to benefit the communica¬ 
tor.” 1 Robotics research is slowly pro¬ 
gressing in this space, with some of 


the earliest work focusing on the evo¬ 
lutionary edge that deceit can provide 
within a group of robotic agents. 2 

Deception and 
Interdependence Theory 

As an outgrowth of our laboratory’s 
research in robot-human trust, where 
robots were concerned as to whether 
to trust a human partner rather than 
the other way around, we considered 
the dual of trust: deception. As any 
good con artist knows, trust is a pre¬ 
cursor for deception, 3 so the transi¬ 
tion to this domain seemed natural. 
We applied the same models of inter¬ 
dependence theory 4 used in our trust 
research and game theory to create 
a framework whereby a robot could 
make decisions regarding both when 
and how to deceive. 5 This involves 
using partner modeling , a simplistic 
view of theory of mind that lets the 
robot assess a situation; recognize 
whether conflict and dependence ex¬ 
ist in that situation between the de¬ 
ceiver and the mark, which indicates 
the value of deception; probe the 
partner (mark) to develop an under¬ 
standing of its potential actions and 
perceptions; and choose an action that 
induces an incorrect outcome assess¬ 
ment in the partner. We implemented 
these results for a simple pursuit- 
evasion task (hide and seek) both 
in simulations and in successfully 
tested robotic systems (see Figure la). 

Changing Strategies 
to Mislead 

Biologists uncovered an interesting 
example in nature regarding decep¬ 
tion’s possible role that involves the 
patrolling strategies squirrels use 
to protect their food caches from 
other predators. 6 Normally, squir¬ 
rels spend time between caches that 
are well stocked. Researchers ob¬ 
served, however, that when a preda¬ 
tor is present (typically, conspecifics 
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Figure 1. Machine and biological deception. We created 
experiments for (a) robot deception based on interdependence 
theory and (b) misleading competitors based on squirrel 
patrolling strategies. 


that are interested in raid¬ 
ing a cache), a squirrel 
will change its patrolling 
behavior to visit empty 
cache sites, with the ap¬ 
parent intent to mis¬ 
lead the raider into be¬ 
lieving those sources are 
where the valuables are 
located, a diversionary 
tactic of sorts. This is a 
form of misdirection in 
which communication oc¬ 
curs implicitly through 
a behavioral change in 
the deceiver. We’ve im¬ 
plemented this strategy 
in simulation 7 and, un¬ 
surprisingly, found that 
these deceptive behav¬ 
iors worked effectively, 
letting robots perform 
better with deception 
than without with respect 
to delaying the time for 
cache discovery. Figure lb 
shows the experimental 
layout for real robots. 

Deception and the 
Handicap Principle 

Steered by our discussions with bi¬ 
ologists, we then investigated the 
handicap principle 8 as a means for 
understanding honest and dishonest 
signaling in animal species. While the 
original formulation of the handicap 
principle stated that all signaling in 
biology must be honest when a suf¬ 
ficiently high cost exists, Rufus John¬ 
stone and Alan Grafen argued that a 
certain level of dishonesty (bluffing) 
could be introduced while preserv¬ 
ing the system’s overall stability in 
the presence of such deceit. 9 This re¬ 
quires a delicate balance of knowing 
when generating such a false signal is 
important and its costs relative to the 
potential success’s value. We explored 
this phenomenon 10 in the context of 
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bird mobbing behavior, which served 
as an original case study for the hand¬ 
icap principle. This model assesses 
the value of a less-than-fit bird (that 
would be prone to capture if set upon) 
joining a mob where group harass¬ 
ment, if sufficiently strong, can lead to 
a predator abandoning an attack. 

Our simulation studies showed 
that deception is the best strategy 
when the addition of deceitful agents 
pushes the mob size to the minimum 
level required to produce enough 
frustration in the predator for it 
to flee. In this case, the predator is 
driven away, and no mob member is 
attacked. For smaller mob sizes, com¬ 
plete honesty yields the lowest mor¬ 
tality rate because the punishment 
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for bluffing is high. If the 
cost of bluffing is re¬ 
duced, adding deception 
can result in a reduced 
mortality rate when the 
predator attacks. Our 
quantitative results ap¬ 
pear elsewhere. 10 We’re 
now importing these sim¬ 
ulation results into our 
robotic platforms for fur¬ 
ther evaluation. 

We’ve successfully dem¬ 
onstrated the value of 
biologically inspired de¬ 
ception in three sepa¬ 
rate cases as applied to 
robotic systems: pursuit 
evasion using interdepen¬ 
dence theory when hiding 
from an enemy; misdirec¬ 
tion based on behavioral 
changes; and feigning 
strength when it doesn’t 
exist. Robotic deception 
is still in its infancy, and 
considerable further study 
is required to make defin¬ 
itive assertions about its 
overall value. This is with 
particular regard to situa¬ 
tions that aren’t simple one-shot de¬ 
ception scenarios, but rather require 
far more sophisticated mental models 
of the mark to sustain deceptive ac¬ 
tivity for longer time periods. 

Serious ethical questions arise re¬ 
garding deception’s role in intelligent 
artifacts that could deceive humans. 11 
Sun Tzu is quoted as saying that “All 
warfare is based on deception,” and 
Machiavelli in The Discourses states 
that “Although deceit is detestable in 
all other things, yet in the conduct of 
war it is laudable and honorable,” so 
it appears a valuable role exists for 
this capability in robotic warfare. In¬ 
deed, an entire US Army Field Man¬ 
ual exists on the subject of deception 
in the battlefield. 12 Nonetheless, when 
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these research ideas and results leak 
outside the military domain, signifi¬ 
cant ethical concerns can arise. We 
strongly encourage further discussion 
regarding the pursuit and application 
of research in deception as applied to 
intelligent machines to assess its risks 
and benefits. 
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Deception in Sports Using 
Immersive Environments 

Sebastien Brault, Richard Kulpa, 

Franck Multon, and Benoit Bideau, 

University Rennes 2 

Why deceive in sports? Basically, to 
exploit an opponent in order to win a 
point—for example, score a goal. In 
sports, various research has reported 
that players frequently use deceptive 
bodily actions, and opponents’ abil¬ 
ity to detect them is a key factor for 
anticipation skill and sports perfor¬ 
mance. 1 ’ 2 Additionally, Robin C. 
Jackson and his colleagues distinguish 
between a player’s attempt to disguise 
his or her potentially informative 
bodily cues by minimizing them and 
an attempt to mislead observers via 
false information (deception). 1 

We’ve explored precisely these sug¬ 
gestions in recent work, 3 showing that 


players minimized the center of mass 
displacement in the medio-lateral plane 
and lower trunk yaw (rotation) during 
deceptive movements in rugby. Con¬ 
versely, they exaggerated outfoot dis¬ 
placement in the medio-lateral plane 
along with head and upper trunk yaw. 
This suggests that rugby players who 
want to win a one-on-one duel must 
take two actions. First, a player should 
use exaggerated body-related infor¬ 
mation to consciously deceive defend¬ 
ers into thinking he or she will run in 
a given direction. Second, the player 
must minimize other postural con¬ 
trol parameters to disguise the sudden 
change in posture necessary to modify 
final running direction. 3 With such an 
attacker’s motor strategy, what might 
be the expert defender’s perceptual 
strategy? Is this defender really influ¬ 
enced by the exaggerated parameters? 
How can we explore these skills in a 
relevant manner? 

Detecting Deception 
Using Virtual Reality 

Jackson and his colleagues didn’t 
make a perceptual analysis of antici¬ 
pation skills in deceptive movement 
in isolation. Their experiments tested 
rugby players’ ability to predict an at¬ 
tacker’s correct running direction as 
early as possible. Results suggested 
that professional players show sig¬ 
nificantly better performance than 
novices in terms of anticipation. To 
explain experts’ high performance, 
we must determine which deceitful 
or disguised information is relevant. 
Virtual reality lets us control the dis¬ 
played deceitful or disguised infor¬ 
mation, such as the virtual charac¬ 
ter’s movement (we discuss virtual 
reality’s advantages over video play¬ 
back elsewhere 3 ). We can then isolate 
this information to determine its in¬ 
fluence on performance. 

We proposed using virtual reality 
for rugby to better understand how 
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people explore visual information 
to predict an opponent’s final run¬ 
ning direction and decide on a mo¬ 
tor response strategy. To this end, we 
stopped the displayed attacking ac¬ 
tions at key moments and looked at 
the correlations between the displayed 
information and the immersed play¬ 
ers’ decision-making (see Figure 2). 

We demonstrated that novices are 
more influenced by exaggerated pa¬ 
rameters than experts, whereas ex¬ 
perts are more attuned to honest sig¬ 
nals. Regarding the defender’s action 
response, we also showed that ex¬ 
perts wait significantly longer than 
novices before initiating a displace¬ 
ment to “intercept” the virtual at¬ 
tacker. Experts can consequently ini¬ 
tiate smaller movement in the wrong 
direction. Another study exposes 
similar results as regards exploring 
the influence of the ball effect on the 
goalkeeper’s hand movement during 
a free kick in soccer. 4 Results showed 
that hand movements were biased in 
the direction of initial ball heading. 
Consequently, during a curved free 
kick, the ball’s initial biased direc¬ 
tion implies larger bias displacement 
for novice goalkeepers compared 
to experts. We can thus view initial 
ball direction as a kind of deceptive 
movement. 

These studies’ main advantage re¬ 
lates to the technology used to im¬ 
merse participants. Contrary to pre¬ 
vious studies using video stimuli to 
explore perception-action in sports, 
more and more projects are based 
on virtual reality. Several sports, in¬ 
cluding soccer, 4 ' 5 rugby, 6-8 and hand¬ 
ball, 9 ' 10 have thus used it. 

Training Deception Skills 
Using Virtual Reality 

Trainers and players have already 
used virtual reality, mainly to im¬ 
prove motor skills such as coordina¬ 
tion in rowing. 11 In a one-on-one duel 



Figure 2. Duel in a virtual environment. We stopped the displayed attacking actions 
at key moments and looked at the correlations between the displayed information 
and the immersed players' decision-making. 


in rugby involving deceptive move¬ 
ments, we can use it to train both the 
attacker and the defender. The de¬ 
fender must pick up relevant infor¬ 
mation on an opponent’s movements 
to react accordingly. Previous experi¬ 
ments show that we can highlight the 
relevant visual information that the 
defender must look at to make an ap¬ 
propriate decision. To this end, vir¬ 
tual reality offers numerous possibili¬ 
ties for focusing the player’s attention 
on the right information at the right 
time. To achieve this goal, we must 
evaluate which type of multi- 
sensory feedback is most appropriate 
in speeding up the learning process. 
For example, using higher- or lower- 
fidelity animated human figures can 
influence the goalkeeper in his or her 
information uptake when trying to 
intercept throws. 10 

To perform an efficient deception 
as an attacker, the player must dis¬ 
guise his or her potential informative 
bodily cues. Virtual reality can give 
the attacker real-time multisensory 
feedback about what bodily cues the 
defender might be able to view. The 
idea is to compute a model based on 
the logistic regression obtained in the 
aforementioned perceptual analysis. 8 
Each logistic regression informs us 
about a predictive percentage of the 
defender’s decision to go to the right 
or the left as regards a given value of 
an attacker’s visual parameter. We can 


then create a model of a defender’s 
decision-making from these real data, 
one for each expertise level. Experts’ 
higher sensitivity, for example, in¬ 
volves a higher percentage of good 
answers with the same quantity and 
quality of information. We can then 
design a virtual defender and use it 
in virtual reality to inform the at¬ 
tacker about the visual body cues that 
the defender can use to detect decep¬ 
tion. However, to ensure that the at¬ 
tacker’s behavior is realistic, we must 
place him or her in an interactive sit¬ 
uation with the defender. The virtual 
defender will then act in real time by 
animating a virtual human accord¬ 
ingly to intercept the user. 

Virtual reality is an interesting tool 
for both understanding the detection 
of deceptive movements and training 
players to perform this detection. It 
nevertheless needs special care during 
setup and use. We must validate at all 
stages—for example, with regard to 
the perception of distance that we 
can alter in virtual environments or 
when modifying the virtual player’s 
movement. For training, we should 
also check whether the skills learned 
in the virtual environment will really 
transfer to the field, and whether they 
last more than a few days. 
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l\lon-Cooperative and 
Deceptive Virtual Agents 

David Traum, University of Southern 
California 

Virtual agents that engage in dia¬ 
logue with people can be used for a 
variety of purposes, including as ser¬ 
vice and information providers, tu¬ 
tors, confederates in psychology ex¬ 
periments, and role players in social 
training exercises. It seems reason¬ 
able that agents acting as service and 
information providers, and arguably 
as tutors, would be truthful and co¬ 
operative. For other applications, 
however, such as role-playing op¬ 
ponents, competitors, or more neu¬ 
tral characters in a training exercise, 
total honesty and cooperativeness 
would defeat the purpose of the exer¬ 
cise and fail to train people in coping 
with deception. The Institute for Cre¬ 
ative Technologies at the University 
of Southern California has created 
several role-playing characters, using 
different models of dialogue and un¬ 
cooperative and deceptive behavior. 
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This article briefly describes these 
models, as used in two different 
genres of dialogue agent: interview¬ 
ing and negotiation. The models 
are presented in order from least to 
most sophisticated reasoning about 
deception. 

Most accounts of pragmatic rea¬ 
soning in dialogue use versions of 
Grice’s cooperative principles and 
maxims 1 to derive utterance mean¬ 
ings (which might be indirect in their 
expression). However, these maxims, 
such as “be truthful,” don’t cover sit¬ 
uations in which conversationalists 
are deceptive or otherwise uncooper¬ 
ative, even though much human dia¬ 
logue contains aspects of uncoopera¬ 
tive behavior. Gricean accounts alone 
don’t adequately cover cases in which 
conversational participants aren’t 
cooperative—for example, why do 
they ever answer at all? The notion of 
discourse obligations 2 differentiates 
the obligation to respond from the 
mechanism of response generation, 
which could be either cooperative, 
neutral, or deceptive. 

Creating Deceptive 
Characters 

The simplest way to create deceptive 
characters is for the scenario author 
to program the deceptive answers di¬ 
rectly into the dialogue policy. In this 
way, the deception comes only from 
the designer, not from the character 
itself, which can’t distinguish decep¬ 
tive from sincere utterances. We’ve 
used this technique for some sim¬ 
ple interviewee characters (including 
C3IT 3 ), where the trainee’s goal was 
to uncover which of several suspects 
was guilty, and deceptive answers 
aided in the diagnosis. 

To engage in more flexible behav¬ 
ior, characters must know the dif¬ 
ference between truthful and decep¬ 
tive or evasive information and be 
able to decide on an honest, evasive, 
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Figure 3. Simulations for tactical questioning and negotiation. We can see human and character interactions with the (a) SASO-ST, 
(b) Tactical Questioning Amani, and (c) SAS04 systems. 


or deceptive dialogue-management 
and response-generation policies. A 
more advanced question-answering 
architecture includes three levels of 
compliance (compliant, reticent, and 
adversarial), and the designer cre¬ 
ates a different response set for each 
of these when creating a character. 4 
When characters are compliant , they 
provide information when asked, but 
fall short of Gricean cooperativity be¬ 
cause they don’t provide helpful in¬ 
formation that was implicated rather 
than explicitly solicited. When char¬ 
acters are reticent, they provide neu¬ 
tral information, but will evade any 
questions about important or sensi¬ 
tive information. When characters 
are adversarial, they provide decep¬ 
tive or untruthful answers. The char¬ 
acters maintain a set of social and 
emotional variables throughout the 
dialogue to determine their com¬ 
pliance level. These include respect 
(trainee for character and charac¬ 
ter for trainee), social bonding, and 
fear. When the variables change sig¬ 
nificantly, the characters change their 
compliance levels, letting the trainee 
experiment with different interview 
strategies, including empathy, threat¬ 
ening, or bargaining. Although these 
characters can reason about when to 
employ deception, the models operate 
globally, depending on the character’s 
mood rather than on the specifics of 
the information itself. When in an 
antagonistic mood, the character will 


lie about everything it can. The sce¬ 
nario author must still decide explic¬ 
itly which content can have deceptive 
answers. 

Tactical Questioning 
and negotiation 

The SASO-ST system provides a vir¬ 
tual reality environment for a trainee 
to practice negotiation (for exam¬ 
ple, convincing a doctor to move his 
clinic). 5 It includes characters who 
have goals that might either align or 
conflict with those of a human dia¬ 
logue participant. When engaged in 
negotiation, the characters will dy¬ 
namically adopt a negotiation strat¬ 
egy, based on a similar, but more 
advanced, calculation of social and 
emotional variables, including trust 
(with subcomponents of solidarity, 
credibility, and familiarity), expected 
utility, and control. Several of these 
strategies are uncooperative, in that 
they try to achieve different outcomes 
than the trainee is trying to achieve. 
In the avoidance strategy, the char¬ 
acters will refuse to answer questions 
germane to the negotiation topic. 
However, they will answer questions 
that are considered irrelevant (mean¬ 
ing questions about issues that aren’t 
related causally to the topic through 
a plan structure). Characters also 
use trust to decide when to believe or 
doubt another character. Moreover, 
characters use the task model struc¬ 
ture to infer implications of what has 


been said, both for Gricean coopera¬ 
tive purposes and to recognize hid¬ 
den motivations that a speaker might 
like to keep private. Figure 3a shows 
negotiations between the character 
“Dr. Perez” and a human playing a 
captain who wants to move the doc¬ 
tor’s clinic. 

The third-generation Tactical Ques¬ 
tioning architecture, 6 ’ 7 allows au¬ 
thors much more fine-grained con¬ 
trol in crafting sophisticated policies 
for uncooperative and deceptive be¬ 
havior than the aforementioned 
models. Using the domain-editor 
software, an author constructs a 
domain-specific ontology of informa¬ 
tion (including false situations) and 
can create specific policies for con¬ 
ditions under which the character 
should be compliant, evasive, or de¬ 
ceptive about any information in the 
ontology. These conditions can in¬ 
clude aspects of the emotional vari¬ 
ables, as before, but also arbitrary 
aspects of the information state, in¬ 
cluding whether specific topics had 
previously been discussed, or whether 
specific incentives have been offered. 
Strategies for several kinds of re¬ 
sponses have been constructed in the 
form of finite-state subdialogue net¬ 
works, which the character can use 
to meet obligations while answering 
(truthfully or deceptively), eliciting 
an offer, or refusing to answer. This 
architecture has been used by stu¬ 
dents and other scenario authors to 
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construct more than a dozen differ¬ 
ent characters for purposes such as 
training tactical questioning (for ex¬ 
ample, the Amani character in Fig¬ 
ure 3b), negotiation, or for use as a 
virtual confederate in psychology ex¬ 
periments. 7 All of these characters 
engage in uncooperative behavior at 
times, and most include some decep¬ 
tive aspects. Two characters in partic¬ 
ular are used for training deception 
detection, 8 where characters’ verbal 
and nonverbal behavior can differ 
along various dimensions. Students 
can learn to follow reliable cues and 
discount unreliable ones. 

Finally, in more recent work on the 
SASO architecture, we have created 
an ability for characters to reason 
more thoroughly about secrecy. In 
the Tactical Questioning architecture 
just described, authors must indicate 
each sensitive piece of knowledge sep¬ 
arately. However, knowledge is often 
related, such that talking about one 
topic might reveal another closely re¬ 
lated one. For the SAS04 scenario 
(see Figure 3c), 9 each virtual charac¬ 
ter has a shameful secret and wants 
to avoid revealing it to the other vir¬ 
tual character. To achieve a successful 
result, the human participants must 
separate the virtual characters and 
only then will the characters bring 
up the sensitive issues, so that the hu¬ 
mans can address them. When the 
participants are all together, the vir¬ 
tual characters must appear to be se¬ 
riously negotiating while keeping se¬ 
cret the shameful reason that they’re 
against the proposal. To achieve this 
with a minimum of authoring over¬ 
head, we have designed and imple¬ 
mented a secrecy inference scheme 
in which the author designates only 
the secret concept and who it must 
be kept secret from. The inference 
scheme then automatically marks se¬ 
cret related actions and states that 
would reveal the secret. Secret items 


won’t be discussed—as proposals, ar¬ 
guments in the negotiation, answers 
to questions, or justifications of other 
claims—if the character that the se¬ 
cret is to be kept from is in contact. 
The preliminary set of inference rules 
is as follows: 

• The sole precondition for a secret 
action is secret. 

• A task with a secret precondition is 
secret. 

• A state that can be achieved only as 
the effect of a secret task is secret. 

• The only task that can establish or 
remove a secret effect is secret. 

These rules are currently at the 
level of heuristics rather than sound 
and complete guarantees of secrecy. 
In particular, the first rule might be 
overly conservative, because other 
reasons might exist for discussing the 
precondition. Moreover, revealing the 
secret assumes that other participants 
have similar task knowledge and in¬ 
ference ability. We have performed 
a preliminary evaluation within the 
SAS04 scenario. We asked eight par¬ 
ticipants to read a brief description of 
the scenario and the main item to be 
kept secret, and then rate all elements 
of the domain as to whether it would 
reveal the secret (76 total concepts, 
15 derived secrets). In all but two 
cases, the majority view agrees with 
the inferences that the system makes 
using the set of aforementioned rules. 
In the two exceptions, there was a 50 
percent split among the participants 
as to whether the concepts were se¬ 
cret (the inference rules mark them 
secret). 

We have briefly presented several 
different architectures for creating 
characters that can engage in nonco¬ 
operative or deceptive dialogue be¬ 
havior. These vary in the authoring 
burden, the characters’ ability to dy¬ 
namically decide on and change their 
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behavior, and their ability to perform 
inference about how sensitive mate¬ 
rial is related. Generally, a trade-off 
exists between simplicity of author¬ 
ing, the amount of authoring needed, 
and the inference ability that the 
character can perform. 
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Deception Detection 
in Multiparty Contexts 

Hayley Hung, University 
of Amsterdam 

Typically, when we consider lie detec¬ 
tion, we think about two people sit¬ 
ting in a room, where one of them 
is probably interrogating the other. 
One person is perhaps attached to 
some sort of device that can mea¬ 
sure physiological changes, while the 
other asks questions and measures 
the physiological responses. This can 
be quite an intimidating and stressful 
situation. 

However, lying or deceiving oc¬ 
curs much more often than we might 
think, and deception or lies might not 
always have a destructive intent. We 
can also view lying as a form of so¬ 
cial lubricant 1 that people use to spare 
someone’s feelings—for example, not 
resolutely turning down a social en¬ 
gagement even when you have no 
intention of going. Under such circum¬ 
stances, lying to prevent unnecessary 
conflict is socially acceptable. Because 
lying is such an embedded part of our 
social activity, we can easily imagine 
several people lying to a single person 
or vice versa over an extended period 
of time or just one meeting. 
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So, deception doesn’t exist just in 
hostile interrogation scenarios or be¬ 
tween just one person and another. 
What about how we actually detect 
deception? Invasive methods that 
measure physiological signals aren’t 
so easy to apply in groups. However, 
researchers have suggested that the 
way we speak and move, which are 
known as nonverbal cues, can be in¬ 
dicative of deception. 1 - 2 Deceiving 
others is a cognitively demanding task 
that involves trying to behave con¬ 
vincingly in one way, while maintain¬ 
ing an awareness of the actual truth of 
the situation. Sometimes, the stress of 
maintaining two different versions 
of reality (the true and the fictitious 
one) can manifest itself as nervous¬ 
ness or subtle differences in behavior. 
Then again, the stress of interroga¬ 
tion can be sufficient for a truth teller 
to behave equally nervously. 1 

Challenges of 
Deception Detection 

The problem with automated meth¬ 
ods of deception detection is that 
stress can manifest itself in many dif¬ 
ferent ways, through either physio¬ 
logical changes or delays in cognitive 
response. However, different people 
handle stress differently, and some 
people can train themselves to sup¬ 
press such feelings. The level of stress 
we feel can also be related to the 
stakes involved in having to lie—if 
the benefit of lying is small, we might 
be less concerned about doing it. 

Deception can occur in groups of 
more than two people and in relaxed 
social settings; our nonverbal behav¬ 
ior indicates whether we’re being de¬ 
ceptive or not, although no clearly 
generalizable set of cues exists. One 
aspect of multiparty deception rarely 
considered forms the basis of the 
work presented here: the hypothesis is 
that in multiparty settings, liars who 
collaborate to dupe others behave 


www.computer.org/intelligent 


differently in their nonverbal behav- 
ior, in when they choose to join a 
conversation, and even in how they 
behave while their “partners in crime” 
are speaking. In these group settings, 
is it more effective to detect decep¬ 
tive behavior automatically using 
cues related to the situation’s social 
context? 

The Werewolf Came 
and Idiap Wolf Corpus 

To investigate this hypothesis, an ex¬ 
periment was devised where groups 
of eight to 12 people sat in a room 
and played a role-playing game. 3 
In the Werewolf Game, players are 
randomly cast as either villagers 
or werewolves, and a games mas¬ 
ter oversees the game, which game 
has two phases. In the night phase, 
all players close their eyes, and the 
wolves surreptitiously indicate to the 
games master which villagers should 
be killed. In the day phase, the play¬ 
ers awake and find out which players 
have been knocked out. Then, the re¬ 
maining “living” players discuss who 
could be a wolf and therefore who 
they should knock out of the game. 
The game continues with alternating 
day and night phases, with a player 
knocked out during each phase. The 
game is over when all the wolves are 
killed or the number of villagers is 
equal to the number of wolves. 

In total, four different groups played 
the game, resulting in 15 games be¬ 
ing played and 81 hours of audio¬ 
visual data for experiments. We used 
a role-playing game because it cre¬ 
ated a natural, nonthreatening en¬ 
vironment in which people could lie 
in a group setting. We anticipated that 
in such a scenario, people would be 
more inclined to get involved and lie 
when necessary. We fitted each player 
with a headset microphone and re¬ 
corded them with video cameras (see 
Figure 4). The data is available at 





Figure 4. Video setup of the Werewolf Game. Three camera views capture the 
visual behavior of eight to 12 people. All views show the players facing the camera 
directly. 
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Figure 5. Extracting cues in speech and 
nonspeech. The example frame shows 
how we extracted regions of motion 
interest from around each person's 
body. FI: head, AS: above shoulders, 
BML/R: body middle left/right, BL/R: 
body left/right. 

www.idiap.ch/scientific-research/ 

resources/wolf-corpus. 

Automatically Extracting 
Nonverbal Cues 

From the audio signal we recorded 
for each person, we extracted de¬ 
rived prosodic features such as 
power, pitch, and fundamental fre¬ 
quency. 3 In addition, we used an 
existing speech/nonspeech detec¬ 
tion system to automatically derive 
a binary vector representing each 
participant’s speaking status. We 
also extracted cues derived from 


each person’s physical motion us¬ 
ing the zones in each frame Figure 5 
shows. 4 The zones were normalized 
by the distance of each person’s chair 
from the camera using the pixel 
width of the chair, and then using the 
frontal face Haar cascade classifier 
implementation from OpenCV to 
automatically determine the zones 
of interest. We extracted audio¬ 
visual cues by accumulating distri¬ 
butions of the visual features during 
periods of each person’s speech or 
nonspeech. 

Detecting Deceptive Roles 

Within the game’s framework, we at¬ 
tempted to identify the werewolves’ 
roles that were explicitly designed to 
deceive the others. We used super¬ 
vised learning methods such as a sup¬ 
port vector machine and relevance 
vector machine, exploiting audio and 
visual features extracted from each 
individual’s behavior to predict play¬ 
ers’ roles. The prosodic audio features 
weren’t very discriminative, leading 
to a worse-than-average estimation 
performance. Using both audio and 
video cues, the performance was bet¬ 
ter, with an above-average chance of 
success. 

When we used the speaking status 
feature and accumulated a feature 
that represented the amount of time 
two people were observed talking 
within a short time window, we ob¬ 
served much more striking differences 
in the werewolves’ behavior com¬ 
pared to that of the villagers. Players 
tended to speak within the same time 


window with different people when 
they were werewolves compared to 
when they were villagers. 

Future Research 

Using social context to detect decep¬ 
tion in multiparty contexts is pre¬ 
liminary work, and further analysis 
is needed to investigate its influence 
on how deceivers behave. Recent 
analysis provides interesting food for 
thought. Figure 6 shows the distribu¬ 
tion of visual activity for both wolves 
and villagers, separated by whether 
they were moving while a different 
wolf or villager spoke. Note the clear 
peak in the probability of low visual 
activity in a wolf when another wolf 
is speaking. Meanwhile, villagers’ 
behavior distributions are identical 
regardless of whether a wolf or vil¬ 
lager is speaking. This suggests a lot 
of potential for identifying subcon¬ 
scious influences that one deceptive 
speaker can have on another. By mov¬ 
ing away from looking at individual 
behavior to the influence that other 
conversants can have on people’s be¬ 
havior, we’ll likely be able to improve 
automated estimates of deception 
from nonverbal cues, performing sig¬ 
nificantly better than humans at the 
same task. 
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Figure 6. Detecting deception in multiparty contexts. We can see 
motion activity for wolves (W) and villagers (V) during the times 
when another W or V was speaking. 
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Deception Detection, 
Human Reasoning, 
and Deception Intent 

Eugene Santos Jr., Deqing Li, 
and Fei Yu, Dartmouth College 

Researchers have found human 
deception-detection skills to be no 
better than random guessing. 1 Al¬ 
though people can learn deception 
cues for specific individuals from past 
successful detections, learning oppor¬ 
tunities for people to enhance their 
capabilities are still scarce. 2 ’ 3 Possible 
ways to improve detection skills in¬ 
clude having a detector focus on spe¬ 
cific individuals and circumstances 
(for example, drug transportation 
through customs checkpoints) or on 
deceptions in a certain problem do¬ 
main (such as accounting fraud). 

Various methods concentrate on 
different communication channels, 
including facial expressions and ob¬ 
servable physiological reactions to 
emotion. However, the prevalent form 
of human interaction has changed 
from the traditional face-to-face 


interaction to computer-mediated 
communication, in which perceivers 
can rarely observe cues in body lan¬ 
guage and facial expressions. Faced 
with this challenge, researchers are 
investigating language-based cues 
based on the intuition that a deceiver 
selects different wording and phras¬ 
ing patterns. In general, deceptive 
stories are often shorter and less com¬ 
pelling than true stories. 4 Deceivers 
tend to use words that are detached 
from themselves, such as “you,” 
“others,” and “human,” whereas truth 
tellers choose words that are closely 
connected to themselves such as “I,” 
“friends,” and “self.” 5 The main chal¬ 
lenge for these methods is that lin¬ 
guistic cues induced in one problem 
domain might not transfer well to 
another. For instance, the deceptive 
reports submitted to a jury can be 
lengthier and even sound more com¬ 
pelling than typical reports. 

In parallel with the development of 
(computational) deception-detection 
methods, considerable effort has also 
been spent on constructing bench¬ 
marking datasets for validation. 
Some datasets are built by collecting 
cases in real life. However, deceiv¬ 
ers might deny their deceptive behav¬ 
ior because of real-life penalties from 


such admissions. Conse¬ 
quently, such datasets typ¬ 
ically lack ground truth. 
If the data is collected in 
a simulated study, testers 
must ensure that the test 
subjects’ deceptive be¬ 
haviors are driven by de¬ 
ceptive intent, which re¬ 
fers to not only the goals 
but also the motivations 
behind them. 6 As such, 
any realistic testbed must 
adequately motivate the 
deceivers-to-be so that 
they are aware of both 
the necessity of deceiving 
and the risk of being detected. This 
all points to the need to account for 
intent in both the development and 
validation of detection methods. 

The Role of Intent 

Our group at Dartmouth College has 
been exploring the role of intent and 
its different uses in deception detec¬ 
tion. 7 ’ 8 We define deception as fol¬ 
lows: In a deceptive communication, 
the information is false from the 
speaker’s viewpoint, the act is inten¬ 
tional, and the purpose is to take ad¬ 
vantage of the listener. The intent of 
deceiving leads to deliberate manip¬ 
ulations and the presupposition of a 
goal. Deliberate manipulations refer 
to manipulating the argument or ob¬ 
servations away from the deceiver’s 
true beliefs, while presupposing the 
goal indicates that all the deceiver’s 
behaviors aim at supporting the goal. 

In our previous work, 8 we success¬ 
fully detected all malicious insider 
threats by detecting changes in the 
cognitive styles of attackers’ written 
reports. Changes in a person’s cog¬ 
nitive styles are reflected as changes 
in his or her underlying syntactic ar¬ 
gument structures and semantic rela¬ 
tion graphs. Our method’s promising 
performance implies that although 
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linguistic content (both syntax and 
semantics) exhibits wide variations, 
we can still quantitatively measure 
cognitive style. Based on this finding, 
we hypothesize that we might also 
be able to reconstruct the underlying 
reasoning process from the linguistic 
content. 

Deception Detection 
through Human Reasoning 

We investigated two major direc¬ 
tions based on reconstructing the 
reasoning process from communica¬ 
tion content. 

Deception Deviates 
from Original Beliefs 

The first direction is to detect deception 
by identifying self-inconsistencies in 
an individual’s reasoning pattern. A 
deception deviates from the deceiv¬ 
er’s original beliefs. Most existing 
studies focus on the search for gen¬ 
eral deception indicators that are ap¬ 
plicable to any individual. Although 
such indicators are effective in eval¬ 
uating deceivers in the same domain 
and might be easy to find and ap¬ 
ply, they aren’t reliable; they might 
be moderated by the environment, 
and outliers can frequently be clas¬ 
sified as deceivers. 7 Compared with 
general indicators, capturing a 
person’s deviation from self isn’t 
straightforward because his or her 
true beliefs are hidden. We’ve ob¬ 
served that acquaintances can bet¬ 
ter anticipate each other’s behaviors 
because they know when they agree 
with each other and when they don’t. 
So, one possible approach is to pre¬ 
dict true beliefs from other truth tell¬ 
ers’ beliefs, given that we know their 
past correlations. We’ve explored and 
implemented this approach in our 
work. 9 

We simulated the opinions of 
a group of agents using Bayesian 
networks (BNs) to model human 


reasoning and learning behavior. 
We predicted agents’ opinions based 
on correlations with each other that 
we obtained from a past history of 
opinions. The system can detect 
87 percent of all self-inconsistencies 
with a false-positive rate as low as 
2 percent. We also validated the self¬ 
inconsistency method using survey 
data in which test subjects were asked 
to lie about their opinions on abor¬ 
tion. 10 To reveal the deviation from 
subjects’ original beliefs, we used 
pairs of deceptive and honest stories 
from the same individuals. We en¬ 
coded the sentiments of some com¬ 
mon arguments into numerical val¬ 
ues to calculate correlations between 
test subjects. Due to the datasets’ lim¬ 
itations, we simulated part of the test 
subjects’ historical opinions based on 
their existing opinions. 

The experimental results showed a 
low inconsistency rate in both honest 
and deceptive stories, meaning that 
only part of the beliefs in deceptive 
stories were inconsistent. If we con¬ 
sider a story to be deceptive when one 
or more of its arguments are incon¬ 
sistent, then the recall rate for detec¬ 
tion is 79 percent, and the precision is 
52.67 percent. Still, we must incorpo¬ 
rate further measurements to refine 
the detection. 

Deception Is an Intentional Act 

The second major direction is to dis¬ 
tinguish misinformation from de¬ 
ception. Inconsistency (consistencies 
with self) and untruthfulness (consis¬ 
tencies with others) indicate decep¬ 
tion. Nevertheless, misinformation 
and other unintentional errors can 
also cause inconsistency and untruth¬ 
fulness. We must distinguish misin¬ 
formation from deception because 
intentional and unintentional distor¬ 
tions of communication are driven by 
different intents and can produce dif¬ 
ferent consequences in the long run. 


Intent to deceive indicates that de¬ 
ception is an intentional act, whereas 
other deviations in behavior, such as 
misinformation, are unintentional 
errors. 

Unfortunately, little research on 
deception-detection realizes the im¬ 
portance of deception being inten¬ 
tional for us to distinguish decep¬ 
tion from unintentional errors. As 
an intentional act, the goal is to 
interact with people in a particu¬ 
lar way for some target reason; for 
deception, that reason is an objec¬ 
tive the deceiver falsifies. Due to 
this unique feature in the deceiver’s 
intent, his or her reasoning process 
when forming the deception also 
becomes unique. 

An experienced deceiver intends 
to maintain the entire story’s con¬ 
sistency, maximize its compelling¬ 
ness, and convey the most support¬ 
ive arguments. 7 While maintaining 
the consistency of the story, the de¬ 
ceiver simultaneously manipulates 
related parts; while maximizing its 
compellingness, the deceiver manip¬ 
ulates some parts of the story more; 
and while conveying supportive argu¬ 
ments, the deceiver emphasizes func¬ 
tional arguments and hides nonfunc¬ 
tional ones. 

We can quantitatively measure the 
reasoning patterns (including argu¬ 
ment structure) using an individual’s 
(structural) inconsistency with him- 
or herself and untruthfulness (struc¬ 
turally) compared against truth 
tellers. From simulated data, we ob¬ 
served these patterns and identified 
that deception and misinformation 
cases are separable. A prototype sys¬ 
tem based on these patterns 7 pro¬ 
duces more reliable results because 
the deceiver’s reasoning process di¬ 
rectly follows the intent to deceive. 
Its unique patterns are usually un¬ 
avoidable and robust to linguistic 
change. 
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A deceiver’s intent plays an impor¬ 
tant role in all aspects of deception 
research, from the development of de¬ 
tection approaches to the construction 
of deception datasets. However, to 
our knowledge, no other study seeks 
insights from the deceiver’s intent, 
formation of deception, or reason¬ 
ing process. Little research in com¬ 
putational methods of deception de¬ 
tection even goes beyond the level 
of words. Understanding the for¬ 
mation of arguments in deception- 
driven reasoning is a rich avenue for 
exploration. 
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Automatic Deception 
Detection in Computer- 
Mediated Communication 

Lina Zhou and Dongsong Zhang, 

University of Maryland, 

Baltimore County 

Deception has changed dramati¬ 
cally since the early 21st century, 
both quantitatively and qualitatively, 
given the advance of information and 


communication technologies. People 
become vulnerable to deception in 
computer-mediated communication 
(CMC), and an urgent need exists to 
increase awareness of such deception 
and the importance of preventing 
and detecting it. However, deception 
detection in CMC faces challenges 
that never existed for detecting face- 
to-face deception. For instance, 
computer-based media generally 
provide restricted modalities for in¬ 
terpreting online interaction, of¬ 
ten limited to text only. In addition, 
computer-based media offer a wide 
range of transmission and processing 
capabilities, each of which warrants 
separate investigations. Evidence 
shows an average person’s accuracy 
of lie-truth discrimination in CMC 
is worse than a random guess. This 
is largely attributed to our limited 
knowledge about cues to online de¬ 
ception as well as human beings’ 
truth bias. Technology can help elim¬ 
inate this bias and be instrumental to 
discovering cues to online deception. 
Thus, using technology support to 
automatically detect online deception 
is particularly promising. 

Automatic Deception 
Detection in CIVIC 

The basic deception detection pro¬ 
cess might involve solving the fol¬ 
lowing problems: cue identification 
that focuses on identifying a set of 
verbal and nonverbal features of 
deceptive communication; cue ex¬ 
traction that deals with extracting 
and encoding those features; decep¬ 
tion reasoning that infers possible 
deception using cues to deception; 
and decision making that involves 
making detection decisions based 
on the inference results as well as 
the user’s or situation’s level of er¬ 
ror tolerance. Here, we discuss the 
first three problems in text-based 
CMC. 
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Identifying Cues from Text 

Interpersonal deception theory is 
widely used to explain online de¬ 
ceptive behaviors. According to this 
theory, a deceiver is engaged in both 
strategic and nonstrategic behaviors. 
Strategic behaviors manifest as plans 
and intentions to achieve deceptive 
goals, whereas nonstrategic behav¬ 
iors reflect the perceptual, cognitive, 
and emotional processes involved in 
deceptive communication. The latter 
behaviors are beyond intentional con¬ 
trol, which we might be able to rely 
on as the basis for identifying cues to 
deception. 

Given that text is the predominant 
medium in CMC, text-based or “ver¬ 
bal” cues have been studied in online 
deception research. Text-based cues 
are related to either the language or 
content of online messages. Com¬ 
pared with content-based cues, which 
refer to messages’ specific informa¬ 
tion features, such as veracity, com¬ 
pleteness, and relevance, language- 
based cues are particularly promising 
because they depend less on expert 
knowledge and prior experience, and 
are relatively insensitive to the do¬ 
main of discourse. 

Researchers have identified 
language-based cues in numerous di¬ 
mensions to date, including quantity, 
nonimmediacy, diversity, specificity, 
language complexity, cognitive com¬ 
plexity, informality, expressivity, af¬ 
fect, and uncertainty. For example, 
deceptive messages in asynchronous 
CMC tend to be longer, more infor¬ 
mal and uncertain, more expressive 
and nonimmediate, less complex, 
and less diverse than truthful mes¬ 
sages. 1 Deception in synchronous 
CMC involves higher levels of cog¬ 
nitive complexity and positive af¬ 
fect than truth-telling. 2 We can mea¬ 
sure each linguistic dimension via 
specific linguistic features or styles. 
For instance, the numbers of words, 


phrases, and messages can indicate 
quantity, and self-references, modal 
operators, and generalizing terms 
represent nonimmediacy. 

Extracting Cues 
via NLP Techniques 

Natural language processing (NLP) 
techniques have benefited and influ¬ 
enced the identification of language- 
based cues. Techniques that have 
been commonly adopted for extract¬ 
ing deception cues include morpho¬ 
logical analysis (determining words, 
nonword tokens, and parts of speech 
in a sentence), shallow syntactic 
parsing (identifying some phrasal 
constituents without indicating their 
internal structures and functions in a 
sentence), and lexical semantic anal¬ 
ysis (interpreting the meaning of 
words without resolving the entire 
sentence’s meaning). The final NLP 
output helps us derive metrics for 
cues to deception. For instance, we 
can measure emotiveness as the ratio 
of the total number of adjectives and 
adverbs to the total number of nouns 
and verbs. A meta-analysis of auto¬ 
matically extracted linguistic cues to 
deception reveals small but signifi¬ 
cant effect sizes for some linguistic 
categories. 3 

Deception Reasoning 
via Machine Learning 

We can treat deception reasoning as 
a classification task, a typical ma¬ 
chine learning problem. While build¬ 
ing models for detecting deception in 
a given context, a classifier can learn 
individual cues’ weights (or impor¬ 
tance) from previously labeled data 
collected from a similar context. We 
can use such models to classify new 
data into deception or truth with a 
certain level of confidence. By em¬ 
ploying training data collected 
from different contexts, we can adapt 
and extend these models to these new 


contexts and associated deception 
strategies. Given that the number of 
instances in the truth class greatly 
outnumbers those in the deception 
class, the development of deception 
reasoning models must address this 
imbalance with the following poten¬ 
tial means: 

• Data sampling. Under-sampling 
the majority (truth) class (randomly 
or selectively) and over-sampling 
the minority (deception) class (dupli¬ 
cates or interpolates) is a classifier- 
independent solution. 

• Parameter tuning. To reduce the 
tendency of standard classifiers in 
learning how to predict the ma¬ 
jority class, we can adjust the cost 
of errors by assigning a higher 
cost to instances of the deception 
class than to instances of the truth 
class. 

• Evaluation metrics. To eliminate 
the bias of accuracy toward the 
truth class, we must also consider 
other performance metrics that fo¬ 
cus on the deception class, such as 
false positives, false negatives, pre¬ 
cision, and recall. 

Researchers have applied various 
machine learning techniques to de¬ 
ception detection, such as neural net¬ 
works, naive Bayes, decision trees, 
support vector machines (SVMs), 
statistical language modeling, and 
neuro-fuzzy methods. Appropriate 
machine learning methods can re¬ 
sult in deception-detection accuracy 
higher than 70 percent if they con¬ 
sider only important cues. 4 Previous 
studies have demonstrated that neu¬ 
ral networks and SVMs are supe¬ 
rior to others in both performance 
and generality. In addition, language 
modeling and neuro-fuzzy techniques 
serve as good alternatives. The for¬ 
mer can capture words’ dependency 
in deceptive communication without 
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explicit feature extraction, and the 
latter addresses uncertainty due to 
imprecision and vagueness of cues to 
deception, their relationships, and the 
outcome of deception detection. 

Future Research Directions 

We’ve made considerable progress in 
online deception detection through 
joint efforts from a wide range 
of disciplines, such as social sci¬ 
ences, communications, computer 
science, and information systems. 
In moving forward, we must build 
upon what’s been achieved and ex¬ 
plore new issues that merit further 
investigation: 

• Analyzing discourse cues to on¬ 
line deception. Online interaction 
is discourse-rich, particularly in 
multiparty communication. Exist¬ 
ing linguistic features have been se¬ 
lected mainly at subsentential lev¬ 
els (such as words and phrases). 
Discourse analysis can potentially 
better reveal the underlying inten¬ 
tion of individual utterances or 
messages by understanding the in¬ 
formation in relation to the dis¬ 
course structure as a whole. This 
is particularly important for un¬ 
derstanding text in CMC that of¬ 
ten diverges significantly from 
traditional formal written text be¬ 
cause the former is relatively brief, 
informal, and poor at manag¬ 
ing interruptions and organizing 
turn-taking. 

• Identifying nonverbal cues to on¬ 
line deception. Nonverbal behav¬ 
ior such as participatory, keyboard, 
social structure, and even brain 
activity will become increasingly 
available in new CMC channels. 
Accordingly, we can identify non¬ 
verbal cues to online deception 
by analyzing interaction, click- 
streams, social networks, senti¬ 
ment, and user brain data. Verbal and 


nonverbal leakage should collec¬ 
tively make powerful cues to on¬ 
line deception. 

• Tracking changes in deception be¬ 
havior. The way deception behav¬ 
ior dynamically adjusts should 
become a source of indirect cues to 
deception. 

• Building standard corpora of on¬ 
line deception. Deception corpora 
remain lacking, particularly for 
real-world online deception, de¬ 
spite a wide variety of decep¬ 
tive acts on the Web. 5 A challenge 
lies in determining the ground 
truth. Active learning is a promis¬ 
ing method for obtaining labeled 
deception data for model training. 
Another is to treat deception detec¬ 
tion as an outlier detection problem. 
Among other deceptive scenar¬ 
ios, online gaming has emerged 
as a new venue for collecting real 
deception data. 

• Incorporating contextual factors 
into deception-detection mod¬ 
els. Cues to deception are likely to 
change as deceivers and receivers 
or communication context (such 
as culture and media synchronic - 
ity) change or deceivers adapt their 
deception strategies. Accordingly, 
deception-detection techniques 
should be adaptive and customized. 

• Improving user satisfaction. In 
addition to the effectiveness of 
deception-detection techniques, 
the success of automatic decep¬ 
tion detection also hinges on 
user satisfaction and system 
usability. 

^Zhallenges in the automatic detec¬ 
tion of deception in CMC will con¬ 
tinue. The field of information and 
communication technology is rapidly 
evolving; so should our knowledge 
about online deception and tech¬ 
niques for deception detection. H 
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